Real-world Effectiveness of BNT162b2 in Children and Adolescents in Preventing Infection and Severe Diseases with SARS-CoV-2 During the Delta and Omicron Periods: Findings from Trial Emulation Using an EHR-based Cohort (preprint)

BACKGROUND The current understanding of the long-term effectiveness of the BNT162b2 vaccine for a range of outcomes across diverse U.S. pediatric populations is limited. In this study, we assessed the effectiveness of BNT162b2 against various strains of the SARS-CoV-2 virus using data from a national collaboration of pediatric health systems (PEDSnet). METHODS We emulated three target trials to assess the real-world effectiveness of BNT162b2: adolescents aged 12 to 20 years during the Delta variant period (Target trial 1), children aged 5 to 11 years (Target trial 2) and adolescents aged 12 to 20 years during the Omicron variant period (Target trial 3). The outcomes included documented infection, COVID-19 illness severity, admission to an intensive care unit (ICU), and two cardiac-related outcomes, myocarditis and pericarditis. In the U.S., immunization records are often captured and stored across multiple disconnected sources, resulting in incomplete vaccination records in patients' electronic health records (EHR). We implemented a novel trial emulation pipeline accounting for possible misclassification bias in vaccine documentation in EHRs. The effectiveness of the BNT162b2 vaccine was estimated from the Poisson regression model with confounders balanced via propensity score stratification. RESULTS During the Delta period, the BNT162b2 vaccine demonstrated an overall effectiveness 98.4% (95% CI, 98.1 to 98.7) against documented infection among adolescents, with no significant waning after receipt of the first dose. During the Omicron period, the overall effectiveness was estimated to be 74.3% (95% CI, 72.2 to 76.2) in preventing documented infection among children, which was higher against moderate or severe COVID-19 (75.5%; 95% CI, 69.0 to 81.0) and ICU admission with COVID-19 (84.9%; 95% CI, 64.8 to 93.5). In the adolescent population, the overall effectiveness against documented Omicron infection was 85.5% (95% CI, 83.8 to 87.1), with effectiveness of 84.8% (95% CI, 77.3 to 89.9) against moderate or severe COVID-19, and 91.5% (95% CI, 69.5 to 97.6) against ICU admission with COVID-19. The effectiveness of the BNT162b2 vaccine against the Omicron variant declined after 4 months following the first dose and then stabilized with higher levels of uncertainty. Across all three cohorts, the risk of cardiac outcomes was approximately 65% to 85% lower in the vaccinated group than that of the unvaccinated group accounting for possible misclassification bias. CONCLUSIONS This study suggests BNT162b2 was effective among children and adolescents in Delta and Omicron periods for a range of COVID-19-related outcomes and is associated with a lower risk for cardiac complications. Waning effectiveness over time suggests that revaccination may be needed in the future.

Challenges of COVID-19 Case Forecasting in the US, 2020-2021 (preprint)

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naive baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making. Author SummaryAs SARS-CoV-2 began to spread throughout the world in early 2020, modelers played a critical role in predicting how the epidemic could take shape. Short-term forecasts of epidemic outcomes (for example, infections, cases, hospitalizations, or deaths) provided useful information to support pandemic planning, resource allocation, and intervention. Yet, infectious disease forecasting is still a nascent science, and the reliability of different types of forecasts is unclear. We retrospectively evaluated COVID-19 case forecasts, which were often unreliable. For example, forecasts did not anticipate the speed of increase in cases in early winter 2020. This analysis provides insights on specific problems that could be addressed in future research to improve forecasts and their use. Identifying the strengths and weaknesses of forecasts is critical to improving forecasting for current and future public health responses.

Excess burden of respiratory and abdominal conditions following COVID-19 infections during the ancestral and Delta variant periods in the United States: An EHR-based cohort study from the RECOVER Program (preprint)

Importance The frequency and characteristics of post-acute sequelae of SARS-CoV-2 infection (PASC) may vary by SARS-CoV-2 variant. Objective To characterize PASC-related conditions among individuals likely infected by the ancestral strain in 2020 and individuals likely infected by the Delta variant in 2021. Design Retrospective cohort study of electronic medical record data for approximately 27 million patients from March 1, 2020-November 30, 2021. Setting Healthcare facilities in New York and Florida. Participants Patients who were at least 20 years old and had diagnosis codes that included at least one SARS-CoV-2 viral test during the study period. Exposure Laboratory-confirmed COVID-19 infection, classified by the most common variant prevalent in those regions at the time. Main Outcome(s) and Measure(s) Relative risk (estimated by adjusted hazard ratio [aHR]) and absolute risk difference (estimated by adjusted excess burden) of new conditions, defined as new documentation of symptoms or diagnoses, in persons between 31-180 days after a positive COVID-19 test compared to persons with only negative tests during the 31-180 days after the last negative test. Results We analyzed data from 560,752 patients. The median age was 57 years; 60.3% were female, 20.0% non-Hispanic Black, and 19.6% Hispanic. During the study period, 57,616 patients had a positive SARS-CoV-2 test; 503,136 did not. For infections during the ancestral strain period, pulmonary fibrosis, edema (excess fluid), and inflammation had the largest aHR, comparing those with a positive test to those with a negative test, (aHR 2.32 [95% CI 2.09 2.57]), and dyspnea (shortness of breath) carried the largest excess burden (47.6 more cases per 1,000 persons). For infections during the Delta period, pulmonary embolism had the largest aHR comparing those with a positive test to a negative test (aHR 2.18 [95% CI 1.57, 3.01]), and abdominal pain carried the largest excess burden (85.3 more cases per 1,000 persons). Conclusions and Relevance We documented a substantial relative risk of pulmonary embolism and large absolute risk difference of abdomen-related symptoms after SARS-CoV-2 infection during the Delta variant period. As new SARS-CoV-2 variants emerge, researchers and clinicians should monitor patients for changing symptoms and conditions that develop after infection.

Risk Factors and Predictive Modeling for Post-Acute Sequelae of SARS-CoV-2 Infection: Findings from EHR Cohorts of the RECOVER Initiative (preprint)

Background Patients who were SARS-CoV-2 infected could suffer from newly incidental conditions in their post-acute infection period. These conditions, denoted as the post-acute sequelae of SARS-CoV-2 infection (PASC), are highly heterogeneous and involve a diverse set of organ systems. Limited studies have investigated the predictability of these conditions and their associated risk factors. Method In this retrospective cohort study, we investigated two large-scale PCORnet clinical research networks, INSIGHT and OneFlorida+, including 11 million patients in the New York City area and 16.8 million patients from Florida, to develop machine learning prediction models for those who are at risk for newly incident PASC and to identify factors associated with newly incident PASC conditions. Adult patients aged  20 with SARS-CoV-2 infection and without recorded infection between March 1st, 2020, and November 30th, 2021, were used for identifying associated factors with incident PASC after removing background associations. The predictive models were developed on infected adults. Results We find several incident PASC, e.g., malnutrition, COPD, dementia, and acute kidney failure, were associated with severe acute SARS-CoV-2 infection, defined by hospitalization and ICU stay. Older age and extremes of weight were also associated with these incident conditions. These conditions were better predicted (C-index >0.8). Moderately predictable conditions included diabetes and thromboembolic disease (C-index 0.7-0.8). These were associated with a wider variety of baseline conditions. Less predictable conditions included fatigue, anxiety, sleep disorders, and depression (C-index around 0.6). Conclusions This observational study suggests that a set of likely risk factors for different PASC conditions were identifiable from EHRs, predictability of different PASC conditions was heterogeneous, and using machine learning-based predictive models might help in identifying patients who were at risk of developing incident PASC. 

Identifying Contextual and Spatial Risk Factors for Post-Acute Sequelae of SARS-CoV-2 Infection: An EHR-based Cohort Study from the RECOVER Program (preprint)

Post-acute sequelae of SARS-CoV-2 infection (PASC) affects a wide range of organ systems among a large proportion of patients with SARS-CoV-2 infection. Although studies have identified a broad set of patient-level risk factors for PASC, little is known about the contextual and spatial risk factors for PASC. Using electronic health data of patients with COVID-19 from two large clinical research networks in New York City and Florida, we identified contextual and spatial risk factors from nearly 200 environmental characteristics for 23 PASC symptoms and conditions of eight organ systems. We conducted a two-phase environment-wide association study. In Phase 1, we ran a mixed effects logistic regression with 5-digit ZIP Code tabulation area (ZCTA5) random intercepts for each PASC outcome and each contextual and spatial factor, adjusting for a comprehensive set of patient-level confounders. In Phase 2, we ran a mixed effects logistic regression for each PASC outcome including all significant (false positive discovery adjusted p-value < 0.05) contextual and spatial characteristics identified from Phase I and adjusting for confounders. We identified air toxicants (e.g., methyl methacrylate), criteria air pollutants (e.g., sulfur dioxide), particulate matter (PM2.5) compositions (e.g., ammonium), neighborhood deprivation, and built environment (e.g., food access) that were associated with increased risk of PASC conditions related to nervous, respiratory, blood, circulatory, endocrine, and other organ systems. Specific contextual and spatial risk factors for each PASC condition and symptom were different across New York City area and Florida. Future research is warranted to extend the analyses to other regions and examine more granular contextual and spatial characteristics to inform public health efforts to help patients recover from SARS-CoV-2 infection.

Machine Learning for Identifying Data-Driven Subphenotypes of Incident Post-Acute SARS-CoV-2 Infection Conditions with Large Scale Electronic Health Records: Findings from the RECOVER Initiative (preprint)

The post-acute sequelae of SARS-CoV-2 infection (PASC) refers to a broad spectrum of symptoms and signs that are persistent, exacerbated, or newly incident in the post-acute SARS-CoV-2 infection period of COVID-19 patients. Most studies have examined these conditions individually without providing concluding evidence on co-occurring conditions. To answer this question, this study leveraged electronic health records (EHRs) from two large clinical research networks from the national Patient-Centered Clinical Research Network (PCORnet) and investigated patients’ newly incident diagnoses that appeared within 30 to 180 days after a documented SARS-CoV-2 infection. Through machine learning, we identified four reproducible subphenotypes of PASC dominated by blood and circulatory system, respiratory, musculoskeletal and nervous system, and digestive system problems, respectively. We also demonstrated that these subphenotypes were associated with distinct patterns of patient demographics, underlying conditions present prior to SARS-CoV-2 infection, acute infection phase severity, and use of new medications in the post-acute period. Our study provides novel insights into the heterogeneity of PASC and can inform stratified decision-making in the treatment of COVID-19 patients with PASC conditions.

Understanding Post-Acute Sequelae of SARS-CoV-2 Infection through Data-Driven Analysis with Longitudinal Electronic Health Records: Findings from the RECOVER Initiative (preprint)

Recent studies have investigated post-acute sequelae of SARS-CoV-2 infection (PASC) using real-world patient data such as electronic health records (EHR). Prior studies have typically been conducted on patient cohorts with small sample sizes 1 or specific patient populations 2,3 limiting generalizability. This study aims to characterize PASC using the EHR data warehouses from two large national patient-centered clinical research networks (PCORnet), INSIGHT and OneFlorida+, which include 11 million patients in New York City (NYC) and 16.8 million patients in Florida respectively. With a high-throughput causal inference pipeline using high-dimensional inverse propensity score adjustment, we identified a broad list of diagnoses and medications with significantly higher incidence 30-180 days after the laboratory-confirmed SARS-CoV-2 infection compared to non-infected patients. We found more PASC diagnoses and a higher risk of PASC in NYC than in Florida, which highlights the heterogeneity of PASC in different populations.

Distributed Counterfactual Modeling Approach for Investigating Hospital-Associated Racial Disparities in COVID-19 Mortality (preprint)

Several studies have found that black patients are more likely than white patients to test positive for or be hospitalized with COVID-19, but many of these same studies have found no difference in in-hospital mortality. These studies may have underestimated racial differences due to reliance on data from a single hospital system, as adequate control of patient-level characteristics requires aggregation of highly granular data from several institutions. Further, one factor thought to contribute to disparities in health outcomes by race is site of care. Several differences between black and white patient populations, such as access to care and referral patterns among clinicians, can lead to patients of different races largely attending different hospitals. We sought to develop a method that could study the potential association between attending hospital and racial disparity in mortality for COVID-19 patients without requiring patient-level data sharing among collaborating institutions. We propose a novel application of a distributed algorithm for generalized linear mixed modeling (GLMM) to perform counterfactual modeling and investigate the role of hospital in differences in COVID-19 mortality by race. Our counterfactual modeling approach uses simulation to randomly assign black patients to hospitals in the same distribution as those attended by white patients, quantifying the difference between observed mortality rates and simulated mortality risk following random hospital assignment. To illustrate our method, we perform a proof-of-concept analysis using data from four hospitals within the OneFlorida Clinical Research Consortium. Our approach can be used by investigators from several institutions to study the impact of admitting hospital on COVID-19 mortality, a critical step in addressing systemic racism in modern healthcare.

Impact of pandemic fatigue on the spread of COVID-19: a mathematical modelling study (preprint)

In late-2020, many countries around the world faced another surge in number of confirmed cases of COVID-19, including United Kingdom, Canada, Brazil, United States, etc., which resulted in a large nationwide and even worldwide wave. While there have been indications that precaution fatigue could be a key factor, no scientific evidence has been provided so far. We used a stochastic metapopulation model with a hierarchical structure and fitted the model to the positive cases in the US from the start of outbreak to the end of 2020. We incorporated non-pharmaceutical interventions (NPIs) into this model by assuming that the precaution strength grows with positive cases and studied two types of pandemic fatigue. We found that people in most states and in the whole US respond to the outbreak in a sublinear manner (with exponent k=0.5), while only three states (Massachusetts, New York and New Jersey) have linear reaction (k=1). Case fatigue (decline in people's vigilance to positive cases) is responsible for 58% of cases, while precaution fatigue (decay of maximal fraction of vigilant group) accounts for 26% cases. If there were no pandemic fatigue (no case fatigue and no precaution fatigue), total positive cases would have reduced by 68% on average. Our study shows that pandemic fatigue is the major cause of the worsening situation of COVID-19 in United States. Reduced vigilance is responsible for most positive cases, and higher mortality rate tends to push local people to react to the outbreak faster and maintain vigilant for longer time.

Unraveling COVID-19: a large-scale characterization of 4.5 million COVID-19 cases using CHARYBDIS (preprint)

Background: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response [1,2]. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) [3] Characterizing Health Associated Risks, and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD.Methods: We conducted a descriptive cohort study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub [4]. Findings: We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts, and are available in an interactive website: https://data.ohdsi.org/Covid19CharacterizationCharybdis/. Interpretation: CHARYBDIS findings provide benchmarks that contribute to our understanding of COVID-19 progression, management and evolution over time. This can enable timely assessment of real-world outcomes of preventative and therapeutic options as they are introduced in clinical practice.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US (preprint)

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. In 2020, the COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized hundreds of thousands of specific predictions from more than 50 different academic, industry, and independent research groups. This manuscript systematically evaluates 23 models that regularly submitted forecasts of reported weekly incident COVID-19 mortality counts in the US at the state and national level. One of these models was a multi-model ensemble that combined all available forecasts each week. The performance of individual models showed high variability across time, geospatial units, and forecast horizons. Half of the models evaluated showed better accuracy than a naive baseline model. In combining the forecasts from all teams, the ensemble showed the best overall probabilistic accuracy of any model. Forecast accuracy degraded as models made predictions farther into the future, with probabilistic accuracy at a 20-week horizon more than 5 times worse than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. f

How the clinical research community responded to the COVID-19 pandemic: An analysis of the COVID-19 clinical studies in ClinicalTrials.gov (preprint)

Objective: The novel coronavirus disease (COVID-19), broke out in December 2019, is a global pandemic. Rapidly in the past few months, a large number of clinical studies have been initiated worldwide to find effective therapeutics, vaccines, and preventive strategies. In this study, we aim to understand the landscape of COVID-19 clinical research and identify the gaps and issues that may cause difficulty in recruitment and the lack of population representativeness. Materials and Methods: We analyzed 2,034 COVID-19 studies registered in the largest public registry - ClinicalTrials.gov. Leveraging natural language processing, descriptive analysis, association analysis, and clustering analysis, we characterized COVID-19 clinical studies by phase and design features. Particularly, we analyzed their eligibility criteria to understand: (1) whether they considered the reported underlying health conditions that may lead to severe illnesses, and (2) if these studies excluded older adults, either explicitly or implicitly, which may reduce the generalizability of these studies in older adults. Results: The 5 most frequently tested drugs are Hydroxychloroquine (N=148), Azithromycin (N=46), Tocilizumab (N=29), Lopinavir (N=20), and Ritonavir (N=20). Most trials did not have an upper age limit and did not exclude patients with common chronic conditions such as hypertension and diabetes that are prevalent in older adults. However, known risk factors that may lead to severe illnesses have not been adequately considered by existing studies. Conclusions: A careful examination of the registered COVID-19 clinical studies can identify the research gaps and inform future COVID-19 trial design towards balanced internal validity and generalizability.